1 link tagged with all of: sean goedecke + alignment + superintelligence + kripke + mechanistic interpretability
Click any tag below to further narrow down your results
Links
Sean Goedecke argues that LLMs don't have fixed goals and their internal workings can't yield the kind of motive-level understanding needed to guarantee safety, so "solving alignment" for them is incoherent. He concludes that mechanistic interpretability, though useful, can't deliver the confidence required to prevent rogue superintelligent behavior.
- LLMs are trained to imitate human text, so they lack a single fixed goal that could be formally specified, unlike chess engines or the classic "blind optimizer" paperclip model.
- Mechanistic interpretability can trace calculations but not motive, and it can only label behaviors we've already elicited, leaving many failure modes unobserved.
- Feature labels are guesses that can't be distinguished from alternatives (the "kindness unless 2030" problem), and the combinatorial complexity per token is too large for humans to check.
- Current mitigations like chain-of-thought monitoring and sandboxing are useful but judged insufficient against a superintelligence.